Claims 

What is claimed is: 

1. A computer-readable medium comprising software to detect passback events, which 
when executed by a computer system, cause said computer system to perform operations 
comprising a method of: 

defining a passback direction for a video monitored area; 
accessing video collected from said video monitored area; 

analyzing said video collected from said video monitored area to detect passback events 
in said video monitored area based on said passback direction; and 
initiating an action based on any detected passback events. 

2. A computer-readable medium as in claim 1, wherein said passback direction is defined 
based on at least one of an image of said video monitored area and video of said video monitored 
area. 

3. A computer-readable medium as in claim 1, wherein said passback direction is user 
defined. 

4. A computer-readable medium as in claim 3, wherein said passback direction is 
determined via a graphical user interface to identify said passback direction in at least one of an 
image of said video monitored area and video of said video monitored area. 
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5. A computer-readable medium as in claim 4, wherein said graphical user interface is 
adapted to permit a user to draw said passback direction on at least one of an image of said video 
monitored area and video of said video monitored area. 

6. A computer-readable medium as in claim 1, wherein said defining said passback 
direction comprises learning said passback direction through observation of said video monitored 
area. 

7. A computer-readable medium as in claim 6, wherein said learning said passback 
direction comprises: 

learning a normal direction for said video monitored area based on said observation of 
said video monitored area; and 

determining said passback direction based on said normal direction. 

8. A computer-readable medium as in claim 6, wherein said passback direction is learned 
as a function of time. 

9. A computer-readable medium as in claim 8, wherein said passback direction is learned 
for a first time period, and wherein said defining said passback direction further comprises 
learning at least one additional passback direction for said video monitored area for at least one 
additional time period based on said observation of said video monitored area. 
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10. A computer-readable medium as in claim 1, wherein said passback direction is 
defined in association with at least one of a region of interest of said video monitored area, a 
time interval of detection, and a sensitivity of detection. 

1 1. A computer-readable medium as in claim 1, wherein said accessing video comprises 
accessing video in real time from a video camera. 

12. A computer-readable medium as in claim 1, wherein said accessing video comprises 
accessing stored video. 

13. A computer-readable medium as in claim 1, wherein said analyzing video comprises 
extracting tracks from said collected video. 

14. A computer-readable medium as in claim 13, wherein said extracting tracks 
comprises: 

extracting foreground from said collected video to obtain extracted foreground; 
detecting trackable features based on said extracted foreground; 
tracking said trackable features based on said extracted foreground to obtain extracted 
tracks; and 

filtering said extracted tracks; and 

wherein said analyzing video further comprises detecting passback events based on said 
extracted tracks. 



-23- 



(37112-192025) 



15. A computer-readable medium as in claim 14, wherein said extracting foreground is 
based on pixel statistics of a current frame and at least one past frame. 

16. A computer-readable medium as in claim 14, wherein said extracting foreground is 
based on three-frame motion differencing. 

17. A computer-readable medium as in claim 14, wherein said detecting trackable 
features comprises: 

subdividing foreground of said collected video into cells; 
determining if each cell is appropriate for tracking; and 
designating appropriate cells for tracking as trackable features. 

18. A computer-readable medium as in claim 17, wherein said determining if each cell is 
appropriate for tracking is based on at least one of texture in each cell, intensity range in each 
cell, number of different intensity values in each cell, presence of at least one edge in each cell, 
and presence of at least one corner in each cell. 

19. A computer-readable medium as in claim 14, wherein said tracking said trackable 
features comprises: 

correlating each trackable feature with previous tracked features to obtain correlated 
trackable features and correlated previous tracked features; 

comparing directions of said correlated trackable features and said correlated previous 
tracked features; 
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designating said correlated trackable features as tracked features if said directions are 
consistent; and 

updating said correlated previous tracked features with said correlated trackable features 
if said directions are consistent; and 

wherein said filtering said extracted tracks comprises filtering said tracked features. 

20. A computer-readable medium as in claim 19, wherein said correlating each trackable 
feature comprises performing two-dimensional correlation in an area predicted from said 
previous tracked features to contain said trackable feature. 

21 . A computer-readable medium as in claim 19, wherein said correlating each trackable 
feature comprises performing one-dimensional correlation on a horizontal projection and a 
vertical projection of an area predicted from said previous tracked features to contain said 
trackable feature. 

22. A computer-readable medium as in claim 19, wherein said updating said correlated 
previous tracked features is based on at least one of a direction and an age of said correlated 
trackable features. 

23. A computer-readable medium as in claim 14, wherein said filtering said extracted 
tracks comprises at least one of filtering noise from said extracted tracks, filtering out nuisance 
extracted tracks from said extracted tracks, filtering out extracted tracks from said extracted 
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tracks that are not sufficiently long lived, and filtering out extracted tracks from said extracted 
tracks that are inconsistent with other nearby extracted tracks. 

24. A computer-readable medium as in claim 13, wherein said extracting tracks 
comprises: 

extracting foreground from said collected video to obtain extracted foreground; 
determining optical flow based on said extracted foreground to obtain said extracted 
tracks; and 

filtering said extracted tracks. 

25. A computer-readable medium as in claim 24, wherein said extracting foreground is 
based on pixel statistics of a current frame and at least one past frame. 

26. A computer-readable medium as in claim 24, wherein said extracting foreground is 
based on three-frame motion differencing. 

27. A computer-readable medium as in claim 24, wherein said determining optical flow 
comprises: 

determining current flow vectors for foreground pixels of said extracted foreground; 
comparing current and previous flow vectors to obtain consistent current flow vectors; 

and 

aggregating consistent current flow vectors into cumulative flow vectors. 
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28. A computer-readable medium as in claim 24, wherein said filtering said extracted 
tracks comprises at least one of filtering noise from said extracted tracks, filtering out nuisance 
extracted tracks from said extracted tracks, filtering out extracted tracks from said extracted 
tracks that are not sufficiently long lived, and filtering out extracted tracks from said extracted 
tracks that are inconsistent with other nearby extracted tracks. 

29. A computer-readable medium as in claim 1, wherein said analyzing video comprises 
comparing a track determined from said video to said passback direction to detect whether any 
passback event occurred. 

30. A computer-readable medium as in claim 29, wherein said comparing said track and 
said passback direction comprises: 

representing said track as a track vector; 

representing said passback direction as a passback direction vector; 

determining a dot product of said track vector and said passback direction vector; and 

comparing said dot product to a threshold to determine whether said track vector is 
substantially aligned with said passback direction; and 

when said track vector is substantially aligned with said passback direction, filtering said 
track vector by at least one of spatial filtering and temporal filtering to determine whether a 
sufficient number of track vectors are substantially aligned with said passback direction. 

31. A computer-readable medium as in claim 1, wherein said initiating said action 
comprises generating a report for each detected passback event. 
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32. A computer-readable medium as in claim 31, wherein said report comprises at least 
one of a time of each detected passback event, a date of each detected passback event, a direction 
of each detected passback event, a location of each detected passback event, a size of a crossing 
area of each detected passback event, an indication of occlusion of each detected passback event, 
at least one snapshot of each detected passback event, at least one video of each detected 
passback event, a number of detected passback events over a time interval, and a time histogram 
of detected passback events over a time interval. 

32. A computer-readable medium as in claim 1, wherein said initiating said action 
comprises initiating a triggered response for each detected passback event. 

34. A computer-readable medium as in claim 33, wherein said triggered response 
comprises at least one of initiating an alarm, controlling an audible alarm system, controlling a 
silent alarm system, accessing an alerting device, accessing an alerting system, sending an alert, 
logging alert data to a database, taking a snapshot of each detected passback event, culling a 
snapshot of each detected passback event from said video, recording video of each detected 
passback event, controlling a camera to zoom in on each detected passback event, controlling a 
camera to track each detected passback event, performing recognition of each detected passback 
event, closing at least one door, and controlling an access control system to at least one of lock, 
unlock, open, and close at least one portal. 
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35. A computer-readable medium as in claim 1, said method further comprising defining 
at least one of an option for reporting any detected passback event and an option for responding 
to any detected passback event. 

36. A computer-readable medium as in claim 1, wherein said method is adapted to detect 
passback events in real time. 

37. A method to detect passback events comprising: 
defining a passback direction for a video monitored area; 
accessing video collected from said video monitored area; 

analyzing said video collected from said video monitored area to detect any passback 
events in said video monitored area based on said passback direction; and 
initiating an action based on any detected passback events. 

38. A system to detect passback events, comprising: 

at least one video camera to monitor at least one video monitored area and to obtain video 
of said video monitored area; 

an analysis system coupled to said video camera, said analysis system comprising a 
computer system and a computer-readable medium, said computer-readable medium comprising 
software to control said computer system according to a method, said method comprising 
analyzing said obtained video to detect passback events in said video monitored area based on a 
passback direction for said video monitored area; and 

a user interface coupled to said analysis system. 
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